Sampling Reflectance Functions for Image-based Relighting

A popular research topic in computer graphics is the acquisition of the appearance of real-world objects. An important aspect of the appearance of an object is how it reacts to incident illumination. The goal of image-based relighting is to visualize real objects under novel incident illumination, and this without explicit knowledge of the object’s geometry or its material properties. The appearance of an object under incident illumination is characterized by its reflectance field. To acquire such a reflectance field, a series of photographs, from a fixed viewpoint, of the object is recorded under different controlled illumination conditions. Applying different illumination conditions to an object is mathematically equivalent to sampling its reflectance field. In the first part of this work, the physical and practical constraints imposed on image-based relighting are studied, and a mathematical framework is derived that encodes these constraints. This framework allows to describe, study, and compare existing relighting techniques, and allows to develop new, more efficient, methods. In the second part, the implications of sampling the reflectance field are studied in detail, and several reconstruction techniques are presented to enhance the visual and numerical quality of the relit results. Additionally, it is shown that an equivalent downsampling operator can be defined on the incident illumination that yields identical results with less computations as the advanced reflectance field reconstruction techniques. Two novel acquisition methods are presented in the third part of this dissertation. These methods differ from other acquisition methods in that they sample a wavelet represented reflectance field directly. The first of these methods samples the reflectance field selectively in the wavelet domain by progressively emitting selected wavelet basis illumination conditions. A feedback loop is used during acquisition to determine what part of the wavelet transformed reflectance field is worthwhile to sample in greater detail. The second technique also samples the reflectance field selectively in the wavelet domain, but decouples the acquisition process and the sampling of the wavelet represented reflectance field. To achieve this, the object is observed under a fixed number of wavelet noise illumination conditions. Afterwards, a progressive algorithm is used to determine the reflectance function of each pixel separately using the observed wavelet noise responses of this pixel. All of the previous techniques are restricted to 2D incident light fields. In the final part of this work, a novel method is presented that is able to relight objects with 4D incident illumination. This allows to visualize objects with spatially varying illumination effects such as spotlights effects and partial shadowing.